{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(page-started)=\n",
    "# Quickstart\n",
    "\n",
    "This notebook quickly demonstrates how you can create biological realistic neural network models that can be used for deep learning.\n",
    "\n",
    "To use Norse effectively, you need to know two things\n",
    "\n",
    "1. How to build neuron models\n",
    "2. How to apply neuron models\n",
    "\n",
    ":::{note}\n",
    "You can execute the code below by hitting <i class=\"fas fa-rocket\"></i> above and pressing <i class=\"fas fa-play\"></i> Live Code.\n",
    ":::"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building Models in Norse\n",
    "\n",
    "Norse is designed to be as simple as possible to use; our neuron models integrate seamlessly with PyTorch - with a single caveat: state.\n",
    "Neurons need to remember their history, similar to recurrent networks in PyTorch. Therefore, we cannot wrap models with the regular [`torch.nn.Sequential`](https://pytorch.org/docs/stable/generated/torch.nn.Sequential.html). \n",
    "\n",
    ":::{seealso}\n",
    "Further details on state handling in {ref}`page-working`\n",
    ":::\n",
    "\n",
    "Our solution is to provide our own [`norse.torch.SequentialState`](https://norse.github.io/norse/generated/norse.torch.module.sequential.SequentialState.html) module, which makes it trivial to construct spiking networks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import norse\n",
    "\n",
    "model = norse.torch.SequentialState(\n",
    "    norse.torch.LIF(),\n",
    "    torch.nn.Linear(2, 1)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how similar this looks to PyTorch's `Sequential`. The only difference is that we have replaced one activation function with a [leaky integrate-and-fire (LIF)](https://norse.github.io/norse/generated/norse.torch.module.lif.LIF.html) model, which is a widely used simple [biological neuron model](https://en.wikipedia.org/wiki/Biological_neuron_model).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Applying neuron models\n",
    "\n",
    "If you are familiar with recurrent networks in PyTorch, you will know that they return not one, but two values: the actual model `output` *and* the model `state` (see e. g. [PyTorch's RNN module](https://pytorch.org/docs/stable/generated/torch.nn.RNN.html#torch.nn.RNN)).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "output, state = model(torch.ones(4, 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Because of their recurrent nature, **neurons exist in time**. A neuron needs time to emit [action potentials/spikes](https://en.wikipedia.org/wiki/Action_potential), so we will always assume that the **outer dimension** of the input tensor is time.\n",
    "\n",
    "Below, we show how you can take a tensor with 4 timesteps (the outer dimension) and apply it to 2 neurons (inner dimension). The linear layer should, then, reduce this to a single neuron simulated over 4 timesteps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[-0.4046],\n",
       "        [-0.4046],\n",
       "        [-0.4046],\n",
       "        [-0.4046]], grad_fn=<AddmmBackward>)"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output, state = model(torch.ones(4, 2))\n",
    "output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Perfect! The output looks as we would expect: it's a `(4,1)` tensor. 4 timesteps, 1 output neuron. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bonus: Applying neuron models with predefined state\n",
    "Before we conclude, note that you can inject a different state if you wish to influence the neuron starting state. Below, we simply use the state we stored above in the `state` variable. The page on {ref}`page-working` will demonstrate how to configure this in depth."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "output, next_state = model(torch.ones(4, 2), state=state)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "This is, honestly, everything you need to get started. Provided you have knowledge with building and training models in PyTorch, you can keep working as you have done so far. If you are new to PyTorch, we highly recommend their suite of tutorials: https://pytorch.org/tutorials/beginner/basics/quickstart_tutorial.html\n",
    "\n",
    "---\n",
    "\n",
    "If you need to simulate more complex settings or configure your own neuron models, please continue to read about {ref}`page-working`, {ref}`page-spiking`, and {ref}`page-learning`.\n",
    "\n",
    "![Norse](../logo.png)"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "1d7c6e90a74384adccd95650de7b60ccf01b88e91c61eab2330936fc6c29e88b"
  },
  "kernelspec": {
   "display_name": "Python 3.8.10 64-bit ('norse': venv)",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": ""
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
